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Abstract. In studying of a class of random neural network, some of relative 
researchers have proposed Markov model of neural network. Wherein 
Markov property of the neural network is based on “assuming”. To reveal 
mechanism of generating of Markov property in neural network, it is 
studied how infinite-dimensional random neural network (IDRNN) forms 
inner Markov representation of environment information in this 
paper.Because of equivalence between markov property and Gibbsian our 
conclusion is that knowledge is eventually expressed by extreme Gibbs 
probability measure —ergodic Gibbs probability measure in IDRNN. This 
conclusion is also applicable to quantum mechanical level of IDRNN. Hence 
one can see “ concept “- “ consciousness” is generated at particle(ion) level 
in the brain and is experienced at the level of the neurons; We have 
discussed also ergodicity of IDRNN with random neural potential. 
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I . Introduction 

In studying of a class of random neural network, some of relative researchers 
have proposed Markov model of neural network (Emile Aarts, Jan Korst, 1989; 
M.M. Van Hulle and G.A. Orban, 1991; Guangcheng Xi, 1991 , and so on). It is 
studied how infinite-dimensional random neural network forms inner Markov 
representation of environmental information in this paper. Representational theory 
of neural network is one how environmental information is modeled in the neural 
network. Forming and renewing of representation of external world in neural 
network involve neural structure. Studying of forming mechanism for this class of 
representation is favourable for understanding location and mechanism of various 
neural structure, the class of representation is base for human brain to generate 


consciousness and can constitute new algorithm or given out solution of some 
algorithms. 

Except for brief introduction, this paper consists of three parts. In second part 
we suppose S$ is countable infinite set of neurons coupled by synapses to one 
another, ( E,B(E£) ) and ( x=<£%,B(x)) two measurable spaces, where E is any 
measurable set of all possible values taking by single neuron. By means of 
analyzing generating process of so-called limit Markov process in IDRNN, we 
have shown in limit state of process for IDRNN to learn of environment (i.e. 
neuron in group of neurons ccs takes a given value y,=y, eg) the limit Markov 
process is generated in IDRNN, this is to say, environment-knowledge is 
expressed by means of the limit Markov process in IDRNN, have pointed out set 
of all stationary distribution of the limit Markov process on IDRNN— set of limit 
Gibbs probability measure is a nonempty closed convex set of which ergodic 
Gibbs probability measure is extreme. Knowledge is eventually expressed by 
extreme Gibbs probability measures in IDRNN. In third part of this paper, we 
have discussed ergodicity of IDRNN with random neural potential. Fourth part is 
a remark. 


II. Markov representation of knowledge in IDRNN 

Suppose S is a countable infinite set of neurons coupled by synapses to one 
another, ( £,8(£)) and ( X =E*,B(X)) two measurable spaces. Where E is any 
measurable set of all possible values taken by single neuron. Hence, we have 
infinite dimensional random neural network (IDRNN). 
In studying of human train’s high function—consciousness, one suggested that a 
key neural mechanism underlying conscious experience are the recentrant 
interactions between posterior thalamocortical areas involved in perceptual 
categorization and anterior areas related to memory, value and planning for action. 
Such interactions among neural groups in distributed brain areas may be necessary 
in order to generate a unified neural process corresponding to a multimodal 
conscious scene (G. Tononi and Edelman. G. M., 1998).In neurophysiological 
research of information representation in the cortex, a possible neural coding 
mechanism of spatiotemporal firing pattern of neurons is based on the temporal 
cell subassemblies which spontaneously dynamically organize into a dynamical 
cell assembly by synchronized activity of these subassemblies.Hebb’s viewpoint 
has considered the assembly of neural cell has played key role in 
neural coding and neural information representatioin. Therefore, in the IDRNN 


by contrast to probability p(x,t) that IDRNN is in state xex at the moment t, we 
have more interested in total probability (or its limit) that neurons in neural group 
ccs take given value y,<¢£,b<ec at the moment t. Such total probability and its 
limit are expressed, respectively, as 

PeOpbEC N= $ pead, (1) 


XYp,p=y, bEC 


BA(Y,bEC)=limpey,.beC.N= J, po. (2) 


xy,=y, bEC 
Suppose for some xex and any xex and /<~, can find finite sequence 
X=X,,X,,*+-,x=x, Such that 
P(X, Xp) P2 X3) PaA) > 5 (3) 

where p(x;,x;)1<i,j</ is transition probability from x, to x,, then there exists limit 

p=limp(x,t, xeXx (5) 
independent of initial distribution p(0). Probability distribution  ={f(o,xe x} is 
called stationary, if p@© =p, then pir) = p0<1r<0. 
Let us suppose that 
(i) The state change of the IDRNN only permits one neuron to change own 
state at a time. 
(ii) The set S of neuron lies in d —dimensional integer lattice R“. AcR? is a 
cube with side 2L in R’, i.e. aea such that a=(a,,a,,---,a,), where a, is integer, 
ja,|< L,i=1,2,.,d . Radius of interaction between neurons is r, i.e., if 
B, ={c:c€A|b—c|<r} is a Sphere with center b, then when x=(y,,ae A) <¢E*, we have 

Pa (2,1) = Po (X Yp) = Pop | Yard E By) » (5) 
if B,ca. Where p,(x,y,) is probability that over time dt IDRNN lying in state x 
changes state of own neuron b from y, to y,. Condition (5) means that probability 
density giving change of state of neuron b is only dependent on states of neurons 
in IDRNN which are not more than r distant from b. 
In later description, the IDRNN will satisfy above-mentioned two conditions. 
For any neuron be R’, density of transition probability p,®,l|y.a€B,) is given and 
satisfies inequality 

0< POs | Yara EB,)<K, (6) 
for all beR’,y, €E, y, €E,aeB, and K<«. Set of all density of transition probability 
{p (l-b eR} is called a family of density of transition probability and denoted by 
A. Restriction on cube Ac R’ of the process {x,@,teT,bert} whose family of 


transition probability is A is a process {x,(#),r¢7,b <A} having finite number of state, 
if B, cA, then when x=(y,,a< 4), following expression 

Pol% Yp) = Php | Yara EB,) xeE*, y,eE , (7) 
holds. When cube A is given, the restriction remains not uniquely be determined. 
Since for those be A,for which distance from b to boundary of cube A is smaller 
than r, possibilities of choosing the density by different manner still will exist. 
Such density is called boundary density, also satisfies following inequality 

0< p, œ% y,) SK, xE E^, y, €E,beA. (8) 
If p,(xy,) only dependent on y,€B,^AbeA), then we will call such boundary 
density local. Most natural method to give local boundary density is to set 

Po Ya) = POs | Yasa EB, NA, ya EE a ER VA) , (9) 
having fixed values y? c£,aer‘\A by any methods. When expression (9) is carried 
out, (y? e E,a eR? \ A) is called constant boundary condition. 
Proposition 1 (Dobrushin R.L.,1971) . 
Suppose for any L=1,2,---- , the restriction of neural process 
{x(a,1),a € R ,t € (set of time)} With family of density a on cube 4, with side 2L is given. 
Let %) =(°,.aeRr*) —arbitrary collection of y? e£ , and p/@|x?),xeX,0<t<0, where 
X, =E“—space of state of the restriction on 4,, be probability of state x on the 
restriction at moment ¢. If initial distribution is chosen so that 

PeOla=L xp =Onaed,eX, 5 (10) 


thus, if Cc A,, y, €E,bec and under inspiration of expression (1) and (2), we have 
Heo beCta= >, pea). (11) 


x:yp =y, dEC 
Then, for any t, 0<t<œ, any finite cer’ and any y,«£,bec, limit 
UČO b EC,t|x1)= lim EO, bE Ctx), (12) 

there exists. 
In IDRNN, set of probability (u2@6,,beC,t|x?),CcR’, y,€E.beC} gives consistent 
family of finite distribution for random field {x(b,),beR4,te7}, and so, in accordance 
with well known Kolmogorov’s theorem, gives probability measure u¢,t|ŝ%) which 
is measurable with respect to %, on measurable space (x = £%,a(x)). Also w(,t| %) 
satisfies following Komogorov-Chapman equation 

1.,t +8 |) = [MGs] Duldit |S) hip EXO <5 <0 <t<w, (13) 


Hence probability measure ,(.r|%,) is transition probability. In accordance with 
Regular Markov existence theorem(M. Loéve, 1978, II), the transition function 
utl) gives time-homogeneous Markov process in the IDRNN. It is limit 
Markov process. 
Now we discuss correctness of expression (12). 
Definition 1. (K.R.Parthasarathy, 1977).A measure „u on a metric space X is said 
to be tight if for each <«>0 there exist a compact set k,c x such that wx-K,)<e«. 
Definition 2. A measurable space (£,8(£)) is called a standard Borel space if there 
exists a metric d on E which turns E into a complete separable metric space and is 
such that gœ) is the Borel o-algebra with respect to d. 
Lemma 1 (K.R.Parthasarathy, 1967). Supposed metric space X is a complete 
separable, then every measure on X is tight. 
In IDRNN, suppose Æ is standard Borel space. Thus measurable space (x = £*" , BX) 
is also standard Borel. From the two definitions 1,2 and Lemma 1, having 
considered subtractness of probability measure (i.e. wx-K,)<« can be expressed 
as X)- uK, <£), we known that there exists a number N such that 

EC - HEC) <2, (14) 
with L>N, i.e., sequence {ué}(L=12,--) converges to ugc), lim weC|)= HEC] + 
From preceding description, we showed in limit state of process for IDRNN to 
learn of environment (i.e. neuron in group of neuron ccs takes a given value 
y,=y,¢£), the limit Markov process is generated in IDRNN. That is to say, 
environment-knowledge is expressed by means of limit Markov process in 
IDRNN. The limit Markov process is a limit behavior of learning process for 
IDRNN . 
Definition 3. For any A, =(al,--,a”)c R1, any x, €£, i=1,2,--,m and any real function 
x(a), ae R? \ A, , probability distribution 

Ag, Œi =1,2,+-,m| x(a) = exp{—BU 4, Œi i =1,.2,+-.m) | xD 4 (x(a) (15) 

is known as Gibbs distribution with boundary condition x(a) in 4,, 
where 

Dy (x)= — Dexp{-BU 4, (4,1 = 12,---,m) | x(a)} (16) 


x, €E,i=l,---m 


is called partition function; 0< #=1/T<«; U, is called neural potential, it is defined 
as 


m 


U4, ((x;,i =1,2,---,m)| x(a) = “0D %; +> Dexa, (a; —a;)+ > S xxo, (a; -—a), (17) 


i=l i=l j=l, j4i i=l aeR“\A, 


where c is a constant, the chemical potential. When L —œ,we have 


lim 4, (+) = AC) = exp{—BU (xi =1,2, 111m] x, ON. (18 ) 
Under neighboring situation, IDRNN is Markovian if and only if its limit 


probability measure is Gibbsian(John G. Kemeny et. al, 1992). By means of the 


Markov process, also i.e. by means of Gibbs probability measure, IDRNN 
expresses environment-knowledge. 


Set of all stationary distribution of the Markov process on IDRNN—set of limit 
Gibbs probability measure is a nonempty compact convex set which ergodic 
Gibbs probability measure is extreme of (H.O. Georgii, 1988). Knowledge is 
eventually expressed by extreme Gibbs probability measure—ergodic Gibbs 
probability measure in IDRNN (Xi Guangcheng, 2003). Assume set of all 
stationary distribution in IDRNN is denoted by M(w),then we have: Suppose 
u,,i=1,++,n, are n extreme points (n—dimesional vector), denote n concepts 


memorized by IDRNN. Then any concept v having n characteristic can be 
uniquelly expressed as convex combination of ,,i=1,-+-,n, 


n 


Ve) Gjhs ia =i. (19) 
i=1 i=1 

Under continuous case, we have 

v= f wold) (20) 
exM (u) 


where we P(exM (1), B(exM (x2), A(.,-) denotes set of all probability measure on 
measurable space (.,.) (Xi Guangcheng, 2003). 


@ The above conclusion is also applicable to quantum mechanical level of 


IDRNN.Suppose that a neuron is excited whenever a quantum particle is 
located inside the neuron.Having given a single particle Hilbert space H ,let 5 be the 


boson Fock space corresponding to IDRNN.For each ye H define a unit vector 
y = 3 , called a coherent state.For space of measure (H ,B(H), o) 


there exists the states pon the Fock space 
p.=sw ><y lody). (21) 
H 
The states pare called classical states and form a convex subset of the state 


space V of s. In certain conditions p is Gibbsian. In addition, the set L of 


linear combination 


DAW, ><W, | > (22) 
{=l 
where A = R (real number) and VY e H „is dense in V for the trace norm 


topology.(E.B.DAVIES, 1976) 
Hence one can see “ concept “- “ consciousness” (Xi Guangcheng, 2003; Xi Guangcheng, 
2007 ) is generated at particle(ion) level in the brain and is experienced at the level of 
the neurons from the above discussion. 


Ill. Ergodicity of IDRNN with random neural potential 


Definition 4. Sequence xu, of probability distribution on X is called converging to 
distribution 7, if for any finite ccr’ and any y, €E,beC, 

lim 14, (2: yp = ypb E CH= WUE: yy = Yp b EC) (18) 
Definition 5. The limit Markov process is called ergodic, if there exists 
probability distribution je) on X such that for any tex, 

lim u(e,t | X) = Ae), (19) 

in sense of definition of converging of expression (18). 
Proposition 2. 
Suppose uv: x >R is a random transformation, R set of all real number; U(x) is 
neural potential function corresponding to x<¢ X . Thus we have probability space 
(U,BU),Q). Let ye,t|%, ex be transition function of the limit Markov process on 
IDRNN. Then there exists transition function gq(e,r|) induced by o such that 
q(e,t |u) = (e,t | x). 
Proof of Proposition 2 relies on following two lemmas. 
Lemma 1(K. R. Parthasurathy, 1977 ). Let X be complete and separable metric 
space s({x})=0,x e€ X;({0,1], B[0.1],) probability space. Thus probability space (x, B(x), 4) is 
isomorphic to ((0,1], B[0,1],/ ). 
Lemma 2(K. R. Parthasurathy, 1977). Let a({y})=0,yer. Then probability space 
(U =R,BWU),Q) is isomorphic to probability space (0.1), B(0,1),/ ). 
From lemma 1 and lemma 2 and having considered transitivity of the 
isomorphism, we find probability space (x, B(x), 4) is isomorphic to the probability 
(U =R,BU),Q) and further find proof of proposition 2. From proposition 2, we have 
following 


Corollary 1. Suppose u ={u,,,.,L@ c Rf, limL@ = R“} is Markov process on probability 


space (U=R,BU),Q) Whose transition function is ge,+|#. Then the limit Markov 
process on IDRNN is ergodic if and only if Markov process 
U = {U 1 L@) c R4, lim L@) = R“} is ergodic. 


Corollary 2. ( Bassalygo,L.A. and Dobrushin,R.L. ) . For IDRNN with 
random neural potential there exist such constants M>0 and m>0, such that if 

OU. > M)<m (20) 
then for 9 —a.e. realization of U, IDRNN with random neural potential v is 
ergodic. 
Proof. From definition of probability distribution o on probability space 
(U =R,BU),Q) and formula (15), the proof is completed. 
Corollary 3. (Bassalygo,L.A. and Dobrushin,R.L.) . For IDRNN with random 
neural potential, if u=(U,,,.(«,,))} satisfies uniformly following formula 

limQ(U,,,, > d)=0 (21) 

for a) and x,,,, then there exists such £, that for almost all #2, and almost 
realization of all neural potential v,,., IDRNN with random neural potential u, 
is ergodic. 
Proof. From corollary 2 and formula (15), the proof is completed. 
By the isomorphism between the state space of IDRNN with random neural 
potential and the space of its potential function, the studying of ergodicity of 
IDRNN with random neural potential is turned into the studying of ergodicity of 
the potential function, the resulting propositions shall be applied to the theory of 
ergodicity of IDRNN with random neural potential. 


IV. Remark 

We note that: 

(1) Probability space  (x.a(x),.) is isomorphic to probability space 
(V,B(V),v ) on the Fock space & ,where V is state space of 
& .Perhaps that is mathematical foundations of Gibbs sameness 
of representation of knowledge at quantum mechanical level 
and level of neurons in IDRNN. 


(2) Let set of linear bounded operator (signal) on the Fock space 
§ be von Neumann algebra ,consider quantum mechanical 
dynamical system C*- dynamic system(A,S(A), a@(R)), where A is von 
Neumann algebra, S(A) is the set of all normal states on 
A, a(R) is a strongly continuous one — parameter group of 
automorphism of A. Let (x,a(x),~ be probability space on compact 
Hausdorff space, xeX >a, €S(A) B(X) —measurable, then A’ 
defined by 


N u= | oud), u EM) 


said to be a channel from M(uw to S(A), and channel A* from S(A) 
to Mi is defined by 

AGC. )=G(m(. ) ) ge S(A), 
where m is positive A-valued measure (POVM) from B(X) to A with 
m(Q)=A(1g) ,Q € B(X), 1, is indicator function(A.S. HOLEVO,1977). 


The theory provided in present article has been used already innovatively 
(Guangcheng Xi and Jianxin Chen,2007). 
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